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Modern TOF-PET scanner systems require high-speed computing re¬ 
sources for efficient data processing, monitoring and image reconstruction. 
In this article we present the data flow and software architecture for the 
novel TOF-PET scanner developed by the J-PET collaboration. We discuss 
the data acquisition system, reconstruction framework and image recon¬ 
struction software. Also, the concept of computing outside hospitals in the 
remote centers such as Swierk Computing Centre in Poland is presented. 


PACS numbers: PACS numbers come here 


1. Introduction 

Positron Emission Tomography (PET) is at present one of the most 
technologically advanced imaging techniques used in medical diagnosis. It 


* Presented at ... 


( 1 ) 



2 


KRZEMIEN PRINTED ON MARCH 1, 2016 


allows for non-invasive tomographic imaging of physiological processes in- 
vivo. The gamma quanta pairs given off by a radioactive tracer administered 
to the patient’s body, are registered in coincidence by the PET scanner 
detector to reconstruct image of the tracer spatial distribution. The sig¬ 
nificant improvement of the image contrast and the faster convergence of 
image reconstruction procedure can be achieved by applying the Time-of- 
Flight (TOF) technique [1112] based on the determination of the annihilation 
point along the Line of Response (LOR) by measuring the time difference 
between the arrival of the gamma quanta at the detectors. 

The current commercial PET devices use inorganic crystal scintillators 
for the detection of the gamma quanta. In contrast, the J-PET collabora¬ 
tion is developing a prototype PET based on polymer scintillators 
10 s n nni EH. This novel approach exploits the excellent time proper¬ 
ties of plastic scintillators, which permit a very precise time measurement, 
making the usage of the TOF technique more effective. The obtained tim¬ 
ing properties allow to extend the J-PET scanner application to studies in 
fields such as material science [I21E3], nano-biology [13] or to investigation 
of fundamental symmetries violation in ortho-positronium system [13 ES]. 

The use of state-of-art detectors together with a dedicated Data Acquisi¬ 
tion System (DAQ) imposes new requirements for the processing of the data 
streams, for the monitoring, as well as for the reconstruction procedures. 
The current prototype of the TOE-PET scanner being developed by the 
J-PET collaboration will consist of 192 of detection modules with double¬ 
sided readout, made out of photomultiplier and front-end electronics (FEE). 
The FEE allows to sample the output time signals at 4 separated voltage 
levels corresponding to 8 samples in total; 4 for signal rising and 4 for falling 
edges. This sums up to 192 x 2 x 8 output channels per event. In addition, 
the front-end electronics work in the so-called trigger-less mode mm, 
storing all incoming events without master-trigger conditions applied. This 
results in a big data flow that needs to be handled and stored efficiently. 
The collected data are processed in several steps (see Fig{^ of low- and 
high-level reconstruction leading to a significant data volume reduction. At 
the same time the information needed to obtain the final image of the hu¬ 
man body is preserved. The process starts with the collection of the raw 
data (time and amplitude are digitized by the time-to-digital and analog-to- 
digital converters). Next, the data is combined into signals and translated 
to hit positions in the individual scintillator modules. Finally, hits in the 
individual detector bars are combined to form LORs. The set of LORs is 
then used as an input for the image reconstruction procedure. To further 
speed up the data processing, several parallelization techniques are applied 
at different stages of computing m- 

In the next sections, we describe in more details the front-end electronics. 
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Fig. 1. Scheme of the data flow in the PET image reconstruction. The raw data 
collected by the Data Acquisition System (DAQ) is then processed by the low- 
level reconstruction module (low-level reco.) and results in the set of reconstructed 
lines of response (LORs), which are then sent to the image reconstruction module 
(Image reco.). The final image can be visualized by the dedicated image viewer or 
can be exported to DICOM format. 


reconstruction framework and image reconstruction techniques developed 
by the J-PET collaboration. We also present the distributed computing 
architecture. 

2. Pront-end electronics 

One of the main novelties of the proposed J-PET detector lays in the 
reconstruction of gamma quanta hit position in the polymer module by 
performing a very precise time measurement. This method puts hard re¬ 
quirements on the read-out electronics. A typical signal rising time from the 
polymer scintillator used in the J-PET project is about 0.5 ns what com¬ 
bined with the rising time of a fast photomultiplier (e.g. R4998) of about 0.7 
ns, results in a signal rising time of about 1 ns. This value allows to obtain 
an excellent time resolution but at the same time the read-out electronics 
must have a much better accuracy to sample those short-time signals e.g. 
probing a signal with a rising time of about 1.2 ns on 16 levels, would require 
an accuracy roughly below 75 ps. Unfortunately, the existing solutions do 
not provide the expected time precision. Therefore, one of our main efforts 
was aimed to construct the read-out electronics which would fulfil the re¬ 
quirements of the detector and at the same time would keep costs reasonably 
low. The design allows for sampling in the voltage domain of signals with 
a duration of few nanoseconds |2fl) . The EEE solution is a purely digital 
implementation, based solely on a EPGA (Eield Programmable Gate Array) 
device and a few satellite discrete electronic components. The input signals 
are amplified and split into four paths, each having an individual thresh¬ 
old level. The design includes only DAG chips (LTC2620) for threshold 
settings and passive splitters connected to the EPGA Low Voltage Differ- 
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ential Signalling (LVDS) buffers. Time to digital conversion is realized in 
the FPGA and it is based on low-delay carry-chains usually used as a part 
of adders. The solution allows to probe the signal in the voltage domain 
with an accuracy below 20 ps (a). The charge of the signal is determined 
by the Time-over-Threshold method (ToT), which is based on the relation 
between the charge and the signal width. An additional advantage of the 
FPGA solution is a very low cost. At present, in the prototype phase, the 
cost together with digitization is only about 10 Euro per sample. The read¬ 
out electronics permit a multi-threshold sampling, which probes the signal 
event waveform with respect to four amplitude thresholds. According to the 
Gompressive Sensing Theory [2T], the collected data points could be used 
to reconstruct the full signal shape e.g. by applying the transformation to 
a sparse representation. The information about the shape of the signals is 
highly correlated with the hit position of the annihilation gamma quantum 
along the scintillator strip. Thus, with this information a better filtering of 
coincidence of the two signals and also a more accurate reconstruction of 
the position are possible [giBlITlES]. 

The FPGA devices are also the core computing nodes of the JPET 
DAQ, which allows for so called continuous or triggerless data taking mode 
in which the data are collected without any central trigger selection condi¬ 
tions HZllIH]. 


3. Low- and high-level data processing 

The raw data provided by the front-end electronics is processed in the 
low-level reconstruction framework, which serves as a backbone system 
for various algorithms (e.g. aforementioned position reconstruction algo¬ 
rithms), calibration procedures and to standardize the common operations, 
e.g: input/output process, access to the detector geometry parameters and 
more [23]. The framework has been developed in C-|—|- using the object- 
oriented approach. It is based on the BOOST [21j and ROOT [2^ program¬ 
ming libraries. The framework is used for the off-line analysis, but also as an 
on-line module being a part of the steering software system PetController. 
The next step in the data processing is the reconstruction of the radioac¬ 
tivity distribution in the patient’s body based on the collected LORs. We 
adopted the most popular approach based on iterative algorithms derived 
from Maximum Likelihood Expectation Maximization (MLEM) |26j. The 
available time-of-flight information is incorporated to improve the accuracy 
and the quality of the reconstruction. The image reconstruction is one of 
the most time-consuming parts of the whole data flow. In order to reduce 
the processing time, parallelization techniques are applied. Currently, a 
solution based on a multi-core CPU architecture is implemented |27j . Ef- 
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forts are made to exploit the processing capability of Graphical Processing 
Units (GPU). The efficient image reconstruction using list-mode MLEM al¬ 
gorithm with approximation kernels was implemented on GPU |28[ 1291IHH] , 
Current J-PET GPU-based reconstruction algorithm is able to provide the 
full 3D reconstruction image of 200^ 2mm voxels in about one minute time, 
by exploiting the time-of-flight information. The comparison between CPU 
and GPU reconstruction time per iteration for the sample Shepp-Logan |31j 
simulated phantom is presented in Figj^ 



8 mm 4 mm 2 mm 

Fig. 2. CPU vs GPU implementation - single image reconstruction iteration time 
as a function of target image resolution. 


4. Remote processing and storage 

Apart from the presented computing schemes, in which the data pro¬ 
cessing is performed locally, using multi-core CPU or/and GPU solutions, 
we consider also a different approach based on remote distributed architec¬ 
ture. The input data (e.g. set of LORs) is not processed locally but is first 
anonymized, encrypted and transferred to the distant computing centre and 
then it is processed by computing nodes of a grid or a cloud network |32]. 
This scenario is a part of the broader computing concept being developed 
by the CIS Swierk Computing Centre. 

The medical data management and storage is not a trivial issue and one 
must consider costs of the maintenance of the specialized servers and backup 
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systems. Also, because of the sensitive character of the medical data the 
security requirements must be fulfilled. In addition, it is expected that the 
overall size of the medical imaging data that needs to be stored will grow 
very fast, and the limited storage space of the local computer resources in 
a medical unit can become an important limitation. 

In the proposed computing model, the processing and the storage of 
medical imaging data are moved from hospitals and healthcare facilities to 
dedicated computing centres, which provide specialized cloud services. The 
hospitals and healthcare centres play a role of clients. As a result, the hos¬ 
pitals are freed from the problem related with the medical data storage, 
backup procedures, or disk space limitations. Also, the centralization pro¬ 
vided by the computing centre improves the level of security through the 
application of the system-wise control and automated security upgrades. 
Last but not least, the overall cost of examination per patient with respect 
to the data processing and further data storage expenses is expected to be 
much lower than in the case of the on-site processing. The other advantages 
of the remote processing model are scalability of the solution and the facil¬ 
ity of data-access. Among the disadvantages of this approach, one can find 
the need of data transfer between the hospital and the computing centre, 
which amounts of about 1 GB per examination, and would increase the to¬ 
tal access time. Also, the remote processing requires per se, the access to a 
reliable transfer link. 

Currently the work on the remote architecture model is ongoing. The 
case of the J-PET scanner is very well suited because of the relatively high 
amount of data per examination generated, due to the triggerless mode of 
operations and the full 3-D field of view. Among the current issues we men¬ 
tion: providing the anonymization of the patients images, the development 
of the data reconstruction algorithms and the efficient data transfer model. 


5. Summary 

We presented the overview of the data processing scheme developed for 
the prototype TOF-PET scanner. The proposed solution is optimized in 
terms of computation time and resources costs. Currently, our work is 
aimed at integration of all components and preparation of the system for 
test measurements with a full-scale prototype. The computing model for 
the remote data processing is being developed. 
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